Let a solve be bounded, and say when Z3 could not decide - #96
Draft
HowardvanRooijen wants to merge 1 commit into
Draft
Let a solve be bounded, and say when Z3 could not decide#96HowardvanRooijen wants to merge 1 commit into
HowardvanRooijen wants to merge 1 commit into
Conversation
Z3Context built its native context from a fixed configuration and offered nothing to add to it: no timeout, no resource limit, no CancellationToken. A theorem Z3 cannot decide - nonlinear integer arithmetic is undecidable in general - ran until the process was killed. And because nothing could bound a solve, Status.UNKNOWN was unreachable and was reported as unsatisfiable. Z3Context gains Timeout and ResourceLimit, applied as parameters on each solver and optimizer so they can change between solves. Every entry point takes an optional CancellationToken, wired to Context.Interrupt; the token is also inspected before the check, because an interrupt that arrives before Z3 starts is lost - measured on the raw API. An UNKNOWN is now an exception. A cancelled token throws OperationCanceledException; anything else - a limit reached, or Z3 giving up - throws TheoremUndecidedException with Z3's reason. The reason strings cannot distinguish the two (the optimizer says "canceled" for both), so the token decides. TrySolve returning false now means only that the theorem was proved to have no solution, which closes the wrinkle #57 raised and #86 had to leave open. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #85.
Settles the third wrinkle of #57.
The defect
Z3Contextbuilt its native context from a fixed configuration and offered no way to add to it.No timeout, no resource limit, no
CancellationToken- andCheck()ran synchronously on thecalling thread. A theorem Z3 cannot decide therefore ran until the process was killed:
And because nothing could bound a solve,
Status.UNKNOWNwas unreachable, soTheorem.csreported it as unsatisfiable - the wrinkle #57 raised and #86 had to leave open.
Measured first, on the raw Microsoft.Z3 API
Before any library change, against the theorem above:
timeout = 500UNKNOWN, reasontimeoutrlimit = 200000UNKNOWN, reasoncanceledtimeout = 500UNKNOWN, reasontimeoutContext.Interrupt()from another thread after 300 msUNKNOWN, reasoninterruptedtimeout = 500UNKNOWN, reasoncanceledUNKNOWN, reasoncanceledSATISFIABLE/UNSATISFIABLEas beforeInterrupt()issued beforeCheck()SATISFIABLE- the interrupt is lostTwo of those rows shaped the design. The reason strings are not consistent - the optimizer says
canceledfor a timeout and an interrupt alike, and an exhaustedrlimitiscanceledon botharrives before the check starts does nothing, so an already-cancelled token has to be inspected
rather than relied on to fire.
The change
Z3Context.TimeoutandZ3Context.ResourceLimit, both nullable, both validated in thesetter, applied as parameters on each solver and optimizer rather than baked into the native
context - so they are properties of the
Z3Contextthat can change between solves, and anundecided solve leaves nothing behind.
ResourceLimitis Z3'srlimit: deterministic, reachedat the same point on every machine, in units with no fixed relationship to time.
A
CancellationTokenon every entry point -Solve,TrySolve,Optimize,TryOptimize,ISolveable<T>, the deferredorderbyform,SolveOrNullandOptimizeOrNull- as an optionalparameter,
CancellationToken cancellationToken = default. A cancelled token callsContext.Interrupt(); the token is also inspected before the check, because of the lost-interruptrow above. What remains is the moment between that inspection and Z3 starting work, which the
remarks say plainly.
How
UNKNOWNis reported, the question #85 deferred to. Put to the maintainer with themeasurements above and chosen over a three-state result:
OperationCanceledException, carrying the token, as everywhere in .NET;UNKNOWN- a timeout, an exhausted resource limit, or Z3 giving up on its own -throws a new
TheoremUndecidedException, carrying Z3's reason string;TrySolvereturningfalsenow means one thing: the theorem was proved to have no solution.Cancellation is recognised from the token rather than the string, because the strings cannot be
trusted to say which it was.
Public surface
Optional parameters on existing methods: source-compatible for every caller, binary-breaking for
the changed signatures.
ISolveable<T>changes shape for the second time in this stack, which2.0 allows and #86 already did.
Z3Contextgains two properties; the library gains one exceptiontype.
The README gets a When Z3 cannot decide section after When there is no solution, with the
theorem above as its example.
Tests
254 → 270, in a new
SolveLimitTests.cs. Every test that runs the undecidable theorem carries a[Timeout], so a regression fails the test rather than hanging the suite - and the whole suitestill finishes in under a second and a half, because the parallel runner overlaps the waits.
Solve_UndecidableTheoremWithATimeout_ThrowsTheoremUndecidedExceptionTrySolve_UndecidableTheoremWithATimeout_ThrowsRatherThanReturningFalseSolve_UndecidableTheoremWithAResourceLimit_ThrowsTheoremUndecidedExceptionrlimit, with no timeout set - so it passing proves the limit reaches the solverSolve_SatisfiableTheoremWithATimeout_StillSolves/TrySolve_UnsatisfiableTheoremWithATimeout_StillReturnsFalseSolve_AfterAnUndecidedSolveOnTheSameContext_StillSolvesSolve_WithAnAlreadyCancelledToken_ThrowsOperationCanceledExceptionSolve_CancelledDuringTheSolve_ThrowsOperationCanceledExceptionOptimize_..WithATimeout../Optimize_CancelledDuringTheSolve..OrderBy_CancelledDuringTheDeferredSolve..SolveOrNull_CancelledDuringTheSolve..Timeout_SetToANonPositiveValue../ResourceLimit_SetToZero../Timeout_SetToNull_ClearsTheLimitNo test asserts on the reason string beyond its presence, for the reason in the table above, and
none asserts on elapsed time.
Mutation results
Chosen so that no mutation can hang the run: the limits themselves are proved by construction
(each limit test sets only the limit it is about), and the mutations target how an
UNKNOWNisreported.
UNKNOWNreported as unsatisfiable again - the pre-#85 codeUNKNOWNis alwaysTheoremUndecidedException; the token never consultedVerification
dotnet build solutions/Z3.Linq.slnx -c Release- clean,TreatWarningsAsErrorsanddocumentation generation on; every new cref resolves
./build.ps1 -Configuration Release- 46 tasks, 0 errors, 0 warningsRelease note
Releases remain on hold under #60 until Microsoft.Z3 5.x reaches nuget.org, so this reaches
mainbut not consumers. Nothing about the hold changes.